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Abstract. Segmentation algorithms based on an energy minimisation framework 
often depend on a scale parameter which balances a fit to data and a regularising 
term. Irregular pyramids are defined as a stack of graphs successively reduced. 
Within this framework, the scale is often defined implicitly as the height in the 
pyramid. However, each level of an irregular pyramid can not usually be readily 
associated to the global optimum of an energy or a global criterion on the base 
level graph. This last drawback is addressed by the scale set framework designed 
by Guigues. The methods designed by this author allow to build a hierarchy and to 
design cuts within this hierarchy which globally minimise an energy. This paper 
studies the influence of the construction scheme of the initial hierarchy on the 
resulting optimal cuts. We propose one sequential and one parallel method with 
two variations within both. Our sequential methods provide partitions near an 
energy lower bound defined in this paper. Parallel methods require less execution 
times than the sequential method of Guigues even on sequential machines. 



1 Introduction 

Despite much efforts and significant progresses in recent years, image segmentation 
remains a notoriously challenging computer vision problem. It's usually a preliminary 
step towards image interpretation and plays a major role in many applications. 

The use of an energy minimisation scheme within the region based segmentation 
framework allows to define criteria which should be globally optimised over a parti- 
tion. Several types of methods such as the Level set [1], the Bayesian [2], the min- 
imum description length [3] and the minimal cut [4] frameworks are based on this 
approach. Within these frameworks the energy of a partition P is usually defined as 
Ex(P) = D(P) + AC(P) where D and C denote respectively the fit to data and the 
regularising term. The energy E A (P) corresponds to the Lagrangian of the constraint 
problem: minimise D(P) subject to C(P) < e. Where 6 is a function of X. Under large 
assumptions, minimising E A (P) is also equivalent to the dual problem: minimise C(P) 
subject to D(P) < e', where e' is also a function of X. Therefore X may be interpreted as 
the amount of freedom allowed to minimise D (D(P) < e') while keeping C as low as 
possible. Since e' is a growing function of X, as X is growing, the constraint on D is more 
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and more relaxed while the importance of the term C is getting more and more impor- 
tant. This parameter A may thus be interpreted as a scale parameter which represents 
the relative weighting between the two energy terms. 

In many approaches the parameter A is fixed experimentally and a minimisation 
algorithm determines for a value of A a. locally optimal partition from the set P of all the 
possible partitions on image /. A sequence of A may also be defined a priori in order to 
compute the optimal partition on each sampled value of A [5]. 

The scale set framework proposed by Guigues [5] is based on a different approach. 
Instead of performing the minimisation scheme on the whole set P of possible partitions 
of an image /, Guigues proposes to restrict the search on a hierarchy H. The advantages 
of this approach are twofold: firstly as shown by Guigues the globally optimal partition 
on H may be found efficiently while the search on the whole set P of partitions only 
provides local minima. Secondly, Guigues shown that if the energy satisfies some ba- 
sic properties, the whole set of solutions on H when A describes R+ corresponds to a 
sequence of increasing cuts within the hierarchy H hereby providing a contiguous rep- 
resentation of the solutions for the parameter A. A method to build the hierarchy H has 
been proposed by Guigues. Since the research space used by Guigues is restricted to the 
initial hierarchy H the construction scheme of this hierarchy is of crucial importance 
for the optimal partitions within H built in the second step. 

This paper explores different heuristics to build the initial hierarchy. These heuris- 
tics represent different compromises between the energy of the final partitions and the 
execution times. We first present in Section 2 the scale set framework. The different 
heuristics are then presented in Section 3. These heuristics are evaluated and compared 
to the method of Guigues in Section 4. 

2 The Scale Set framework 

Given an image / and two partitions P and Q on /, we will say that P is finer than Q 
(or Q is coarser then P) iff Q may be deduced from P by merging operations. This re- 
lationship is denoted by P<Q. Let us now consider a theoretic segmentation algorithm 
Pa parametrised by A. We will say that P is an unbiased multi-scale segmentation algo- 
rithm iff for any couple (A\, Ai) such that A\ < A2, and any image /, P^ (I)<P^ 2 (I). If Px 
is an unbiased multi-scale segmentation algorithm, P\(l) increases according to A and 
the set H = UieR+ PxiT) defines a hierarchy as an union of nested partitions. Note that 
the set P of partitions on / being finite, H must be also finite. 

Unbiased multi-scale segmentation algorithms follow a well known causal princi- 
pal: increasing the scale of observation should not create new information. In other 
words any phenomenon observed at one scale should be caused by objects defined at 
finer scales. In our framework, increasing the scale should not create new contours. 

The family of energies considered by Guigues corresponds to the set of Affine 
Separable Energies (ASE) which can be written for any partition P of / in n regions 
{R u ...,R n } as: 



n n n 

E(P) = D(P) + AC(P) = J] D(Rd + A J] C(Rd = J] D(tf ,■) + AC(Rd 
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Let us consider a hierarchy H and the sequence (C^(//))agr+ of optimal cuts within 
H. The approach of Guigues is based on the following result: If E A (P) is an ASE and if 
C A (P) is decreasing within P: 

V(P,0gP P<Q => C(P) > C(Q) 

then the sequence (C* A (H)) Ae -^+ is an unbiased multi-scale segmentation. The union of 
all (C^(ZO)agr+ defines thus a new hierarchy within H. The tree corresponding to the 
hierarchical structure of UieiR+ C\(H) may be deduced from H by merging with their 
fathers all the nodes which do not belong to any optimal cuts. Note that an equivalent 
result may be obtained if no condition is imposed to C but if D is increasing according 
to A. 

The restriction by Guigues of the research space to a hierarchy may thus be justified 
by the fact that the set of partitions produced by any unbiased multi- scale segmentation 
algorithm describes a hierarchy. Conversely, given a hierarchy //, if the energy E A is 
an ASE with a decreasing term C the sequence of optimal cuts of H according to E A : 
(C* l (H)) / ieR+ is an unbiased multi-scale segmentation algorithm. 

Given a partition P e P, the decrease of C may be equivalently expressed as a 
sub-additivity relationship: 

V(fl, R')eP\Ris adjacent to R' C(R U R') < C(R) + C(R') (1) 

Note that the sub-additivity of the regularising term C in common is many applica- 
tions. For example, if C is proportional to some quantity summed up along contours, C 
is sub-additive due to the removal of the common boundaries between the two merged 
regions. Moreover, the term C may be interpreted within the Minimum Description 
Length framework [3] as the amount of information required to encode a partition. 
Therefore, one can expect C to decrease when the partition gets coarser. 

Given a hierarchy H, the sequence of optimal cuts C* A (H) within H has to be com- 
puted. Let us consider one region R at the second level of the hierarchy (computed from 
the base) and its set of sons S \ , . . . , S n . Let us additionally consider the tree H(R) rooted 
at R within H (Fig. 1(a)). Since R is a level 2 node, the hierarchy H(R) allows only 
two cuts: one encoding the partition P\ made of the sons of R whose energy is equal to 
Ea(P\) = Z?=i^0S/)+^Z?=i COS;) and one encoding the partition/^ reduced to the sin- 
gle region R. The energy of P2 is equal to E A (P 2 ) = D(R) + AC(R). Due to the sub addi- 
tivity of C we have YJl=\ C(Ri) > C(R). Therefore, using the linear expression of E A (P\) 
and E A (P 2 ) in A, if J£i D(S t ) < D(R) the line E X (P X ) = D(S t ) + A^U C(R t ) is 
below the line E A (P 2 ) = D(R) + AC(R) until a value A + (R) of A for which the two lines 
cross(Fig. 1(b)). If D(Si) > D(R), E A (P 2 ) is always greater or equal to E X (P\) in 
which case we set A + (R) to 0. Therefore, in both cases the partition P\ is associated to 
a lower energy than P 2 for A = until A = A + (R). Above this value the partition P 2 is 
associated to the lowest energy. In terms of optimal cuts, P\ corresponds to the optimal 
cut of H(R) until A + (R) and P 2 is the optimal cut above this value(Fig. 1(c)). The value 
A + (R) is called the scale of appearance of the region R. 

Guigues shown that the above process may be generalised to the whole tree. Each 
node of H is then valuated by a scale of appearance. Some of the nodes of H may 
get a greater scale of appearance than their father. Such nodes do not belong to any 
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A + (R) AAA 
A+(R) 



(a) H(R) (b) E X {P X \ E A (P 2 ) (c) E X {C\{H{R))) (d) E A (C* A (H)) 

Fig. 1. (a) a node of the hierarchy whose sons {Si,...,S n } correspond to initial regions, (b) 
the energies of the partitions associated to R and [Su...,S n ] plotted as functions of A. (c) the 
energy of the optimal cuts within H(R) (a), (d) an example of concave piecewise linear function 
encoding the energy of the optimal cuts within a global hierarchy H. 



optimal cut and are removed from H during a cleaning step which merges them with 
their fathers. Each node R of the resulting hierarchy belongs to an optimal cut from 
A - A + (R) until the scale of appearance of its father A + QF(R)), where T{R) denotes the 
father of R in H. The value A + (R) may be set for each node of the tree using a bottom-up 
process. The optimal cut C* A (H) for a given value of A may then be determined using 
a top-down process which selects in each branch of the tree the first node with a scale 
of appearance lower than A. The set of selected nodes constitutes a cut of H which 
is optimal by construction according to E A . The function E A (C* A (H)) corresponds to a 
concave piecewise linear function whose each linear interval corresponds to the energy 
of an optimal cut within H (Fig. 1(d)). 

Given a hierarchy H and the function E A (C* A (H)) encoding the energy of the se- 
quence of optimal cuts, the optimality of H may be measured as the area under the 
curve E A (C* A (H)) for a given range of scales or as the area of the surface A (Fig. 1(d)) 
between E A (C* A (H)) and the energy of the coarsest cut E A (P max ). Where P max denote 
the partition composed of a single region encoding the whole image. We propose in 
Section 4 an alternative measure of the quality of a hierarchy which allows to reduce 
the influence of the initial image. 

Guigues proposed to build a hierarchy H by using an initial partition Pq and a strat- 
egy called the scale climbing. This strategy merges at each step the two adjacent regions 
R and R' such that: 

= D(RVR')-D(R)-D(R') = D(R l U R 2 ) - D(Ri) - D(R 2 ) 

C(R) + C(R') - C(R U R') (r u r 2 )^p\r^r 2 C(Ri) + C(R 2 ) - C(Ri U R 2 ) 

(2) 

where P denotes the current partition and R\ ~ R 2 indicates that R\ and R 2 are adjacent 
in P. 

This process merges thus at each step the two regions whose union would appear 
at the lowest scale. Such a construction scheme is coherent with the further processes 
applied on the hierarchy. However, there is no evidence that the resulting hierarchy may 
be optimal according to any of the previously mentioned criteria. We indeed show in the 
next section that other construction schemes of a hierarchy may lead to lower energies. 
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3 Construction of the initial hierarchy 

Many energies have been designed in order to encode different types of homogeneity 
criteria (piecewise constant [3,6], linear or Polynomial [3] variations,. . .). This paper 
being devoted to the construction schemes of the hierarchy, we restrict our topic to the 
piecewise constant model described by Leclerc [3] and Mumford and Shah [6]. The 
energy of this model may be written as: 



where P = {R\, . . . , R n ] represents the partition of the image, SE(Ri) = J] peR \\c p - ji R \\ 2 
is the squared error of region R t and \S(Ri)\ is the total length of its boundaries. 

Within the Minimum Description Length framework, S E(Ri) may be understood 
as the amount of information required to encode the deviation of the data against the 
model, while \6(Ri)\ is proportional to the amount of information required to encode 
the shape of the model. Within the statistical framework, the squared error may also 
be understood as the log of the probability that the region satisfies the model (i.e. is 
constant) using a Gaussian assumption while \6(Ri)\ is a regularising term. 

Our approach follows the scale climbing strategy proposed by Guigues (equation 2). 
Given a set W of regions within a partition P we thus consider the scale of appearance 
of the region R defined as the union of the regions in W. The heuristics below use this 
basic approach but differ on the sets W which are considered and on the ordering of the 
merge operations. 

3.1 Sequential Merging 

Given a current partition P, let us consider for each region R of P, its set V(R) defined 
as {R} union its set of neighbours and the set P*(V(R)) of all possible subsets of V(R) 
including R. Each subset W e *P*(V(R)) encodes a possible merging of the region R 
with at least one of its neighbour. Let us denote by R w = Ur^wR' me region formed 
by the union of the regions in W. Note that the region R w is connected since R belongs 
to W and all the regions of W are adjacent to R. Let us additionally consider the two 
partitions of R w \ P R w = {R w } and Pw = W. The energies associated to these partitions 
are respectively equal to E A (P R w) = D(R W ) + AC(R W ) and: 



where D(W) and C(W) denote respectively the fit to data and the regularising terms of 
the partition P w . 

Since C is sub additive (equation 1) we have C(W) > C(R W ). The energy E A (P W ) is 
thus lower than E A (P R w) until a value A + (R W ) called the scale of appearance of R w (Sec- 
tion 2). Using the scale climbing principle, our sequential merging algorithm computes 
for each region R of the partition the minimal scale of appearance of a region R w \ 



n 




(3) 



E A (P W ) = D(W) + AC(W) = Yj D ( R ') + X Yj C(R/) 



R'eW R'eW 



A min( R ) = argmin We <p* (v(R)) 



D(R W ) - D(W) 
C(W) - C(R W ) 
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the set W e P*(V(R)) which realises the min is denoted W min (R). 

Given the quantities A^ in (R) and W min (R), our sequential algorithm iterates the fol- 
lowing steps: 

1. Let P denotes the current partition initialised with an initial partition Pq, 

2. For each region R of P compute A^ in (R) and W m i n (R) 

3. Compute R min = arg min Re pA^ in (R) and merge all the regions of W min (R min ). 

4. If more than one region remains go to step 2, 

5. Output the final hierarchy H encoding the sequence of merge operations. 

This algorithm performs thus one merge operation at each step of the algorithm. 
Note that all the regions of W m i n (R m i n ) are adjacent to R m i n . Therefore, within the irreg- 
ular pyramid framework, the merge operation may be encoded by a contraction kernel 
of depth one composed of a single tree whose root is equal to R min . The computation of 
A^faiR) for each region R of the partition requires to traverse *P*(V(R)) whose cardinal 
is equal to 2 |y(/?)l_1 . Therefore, if the partition is encoded by a graph G = (V, E), the 
complexity of each step of our algorithm is bounded by V|2 fc ) where | V| denotes the 
number of vertices (i.e. the number of regions) and k represents the maximal vertices 's 
degree of G. The cardinal of V is decreased by \W m i n (R m in)\ - 1 at each iteration. Since 
W m in(Rmin)\ * s at l east equal to 2, the cardinal of V decreases by at least 1. The com- 
putation of A^ in (R) for each region R of the partition may induce important execution 
times when the degree of the vertices of the graph is important. However, experiments 
presented in Section 4 show that the cardinal of the subsets W e P*(R) may be bounded 
without altering significantly the energy of the optimal cuts. Let us finally note that this 
algorithm includes the scale climbing approach proposed by Guigues. Indeed, the merge 
operations studied by Guigues (Section 2) correspond to the subsets W e P*(V(R)) with 
| W\ = 2 which are considered by our algorithm. 

3.2 Parallel Merge algorithm 

Our parallel merge algorithm is based on the notion of maximal matching. A set of 
edges M of a graph G = (V, E) is called a maximal matching if each vertex of G is inci- 
dent to at most one edge of M and if M is maximal according to this property. Moreover, 
we would like to design a maximal matching M such that the scale of appearance of the 
regions produced by the contraction of M is as low as possible. Let us denote by i(e), 
the two vertices incident to e. Using the same approach as in Section 3.1 we associate to 
each edge e of the graph the scale of appearance A + (i(e)) (equation 2) of the region R L ^ 
defined as the union of the regions encoded by the two vertices incident to e. Following, 
the same approach as Haxhimusa [7] we define our maximal matching as a Maximal 
Independent Set on the set of edges of the graph. The iterative process which builds 
the maximal independent set selects at each step edges whose scale of appearance is 
locally minimal. This process may be formulated thanks to two boolean variables p and 
q attached to each edge such that: 



pi = A + (e) = mm e , er(e) {A + (e')} J p k e +l =p k e V (q k e A A + (e) = min e , er(e) , ^ [A + {e')}) 





(4) 



Scale Set representation for image segmentation 



7 



where r(e) denotes the neighbourhood of the edge e and is defined as r(e) = {e} U {e' e 
E\i{e) n i{e') * 0}. 

This iterative process stops when no change occurs between two iterations. If n 
denotes the final iteration, the set of edges such that p n e is true defines a maximal match- 
ing [7] M which encodes the set of edges to be contracted. Moreover, the set of selected 
edges corresponds to local minima according to the scale of appearance A + (e). Roughly 
speaking if A + (e) is understood as a merge score, one edge between two vertices will be 
marked (p k e = true) at iteration k, if among all the remaining possible merge operations 
involving these two vertices, the one involving them is the one with the best merge 
score. Note that the construction of a maximal matching is only the first step of the 
method of Haxhimusa which completes this maximal matching in order to get a deci- 
mation ratio of order 2. The restriction of our method to a maximal matching allows to 
restrict the merge operations to edges which become locally optimal at a given iteration. 
We thus favour the energy criterion against the reduction factor. As shown by Bield [8], 
the reduction factor in terms of edges induced by the use of a maximal matching is a 
least equal to 2|^y where k is the maximal vertex's degree of the graph. The edge's 
decimation ratio may thus be very low for graphs with important vertices 's degrees. 
Nevertheless, experiments performed on 100 natural images of the Berkeley database 1 
shown that the mean vertex's decimation ratio between levels on this database is equal 
to 1.73 which is comparable to the 2.0 decimation ratio obtained by Haxhimusa. 

The local minima selected in equation 4 are computed on decreasing sets along the 
iterations in order to complete the maximal matching. We can thus consider that the 
detected minima are less and less significants as the iterations progress. We thus pro- 
pose an alternative solution which consists in contracting at each step only the edges 
selected at the first iteration (p l e = true). These edges correspond to minima computed 
on the whole neighbourhood of each edge. This method may be understood as a combi- 
nation of the method proposed by Haxhimusa [7] and the stochastic decimation process 
of Jolion [9] which consists in merging immediately vertices corresponding to local 
minima. 



4 Experiments 

The different heuristics presented in this paper have been evaluated on the Berkeley 
database. The evaluated heuristics include our parallel merge heuristic based on a max- 
imal matching (MM) and the variation of this methodCMM 1 ) which merges at each step 
the edges selected during the first iteration (Section 3.2). We also evaluated our sequen- 
tial method (SM) and two variations of this method: the first variation (S M 2 ), considers 
for each region R of the partition the subsets of cardinal 2 of V(R). This method corre- 
sponds to the heuristic proposed by Guigues. We also evaluated an intermediate method 
(S M 5 ) which restricts the cardinal of the subsets of V(R) including R to an upper thresh- 
old fixed to five in these experiments. All the experiments have used an initial partition 
obtained by a Watershed algorithm [10]. 



1 available at http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/ 
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A = 0.2 A = 0.4 A = 0.6 A = 0.8 A = 0.2 A = 0.4 A = 0.6 A = 0.8 



Fig. 2. Partitions of the mushroom and the fisherman images at different scales. Each line of the 
array corresponds to an heuristic whose acronym is indicated on the first column. 



Fig. 2 shows 5 optimal cuts obtained for increasing values of A on the Mushroom 
and Fisherman images of the Berkeley database 1 . The heuristics used to build the hier- 
archies are displayed on the first column of Fig. 2. The original images are displayed in 
Fig. 4(a). 

Fig. 3(a) shows the influence of the number of initial regions on the execution time. 
These curves have been obtained on the Mushroom image with different initial parti- 
tions obtained by varying the smoothing parameter of the gradient within our Watershed 
algorithm. 

Fig. 3(b) allows to compare the performance of each heuristic on the whole Berke- 
ley database. However, a direct comparison of the energies obtained by the different 
heuristics on different images would be meaningless since the shape of the function 
E A (C* A (H)) depends both of the intrinsic performances of the heuristic used to build H 



1 Color plates are available at the following url: http://www.greyc.ensicaen.fr/~jhpruvot/Cut/ 
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(a) Execution time (b) E A (C* A (H)) (c) Energy's Bounds 

Fig. 3. (a) execution times of the different heuristics on the Mushroom image (Fig. 2) using an 
initial partition with a varying number of regions, (b) mean energies of optimal cuts obtained by 
our heuristics on the Berkeley database, (c) bounds of the optimal cut's energies. 



and of the image / on which H has been built. We have thus to normalise the energies 
E A (C* A (H)) produced by the different heuristics before any comparison. 

Given a hierarchy H, since C* A (H) is an unbiased multi-scale segmentation (Sec- 
tion 2), the hierarchy H obtained by each of our methods may be associated to a value 
^max a bove which the optimal partition P max is reduced to a single region encoding the 
whole image. The energy of P max is defined as: E A {P max ) = D\ + AC\ where D\ - SE(I) 
denotes the global image's squared error and C/ = \S(I)\ the perimeter of the image. 
Since the energy of the optimal cuts E A (C* A (H)) of a hierarchy H is a piecewise linear 
concave function of A, the function E A (C* A (H)) is below the energy E A (P max ) associated 
to the coarser partition(Fig. 3(c)). Moreover, if Pq denotes the initial partition, the two 
points (0,Eq(Pq)) and (A max , E Amax (P max )) belong to the curve. Therefore, E A (C* A (H)) 
being concave, it should be above the line connecting these two points. Finally, the 
line connecting (0, 0) to (A max , E Amax (P max )) being below the line joining (0, Eo(Pq)) and 
(A max , E Amax (P max )) we have for any hierarchy H and any scale A (Fig. 3(c)): 



E Amax (P max ) < E A {C\{H)) < E A (P max ) 

^max 

We obtain from this last inequality and after some calculus the following equation: 

v/i^t^ ^ , , ~ 1 ^ E X (C\{H)) A A max Ci 

VA e R+ x A < 1 + - — < — — - — - < 1 with x A = - — and E f = — — (5) 

1 + X A ti E A (r max ) A max L>i 

E (C* (//)) j 

Therefore, using the normalised energy, | — - and the normalised scale x A = 

any curve -^-^ — r- lies in the upper left part of the unit cube [0, 1] . Note that this result 
is valid for any hierarchy H and thus any heuristic. 

Using our piecewise constant model (equation 3), the energy E A (P max ) is roughly 
equal to the squared error of the image for small values of A and may be interpreted 
as the global variation of the image. The normalised energy allows thus to reduce the 
influence of the global variation of the images on the energy and to compare energies 
computed with a same heuristic but on different images. Note however, that the use of 
the normalised scale x A = j^- discards the absolute value of A max . We thus do not take 
into account the range of scales for which the optimal cut is not reduced to the trivial 
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(a) Original Images 




(b) D(R) = SE(R) 



(c) D(R) = SE(R)(1 + 

f ( IntjR) x 
J ^ Ext(R) > 



Fig. 4. (a) Original images, (b) and (c), partitions of the tower image built with a same heuris- 
tic(SM) at a same normalised scale (x A = .8) but with energies defined using two different fit to 
data terms, (b) is defined using the squared error D(R) = S E(R) while (c) is defined using the 
formula defined by equation 6. 



partition P max . However, the absolute value of A max varies according to each image and 
each heuristics. The normalised scale allows thus to remove the influence of the image. 
Moreover, our experiments shown thus that for each image, our different heuristics 
obtain close A max values. 

Fig. 3(b) represents for each value of x A and each heuristic, the mean value of 

E (C* (HY) 

the normalised energy — - computed on the whole set of images of the Berckley 
database. 

As shown in Fig 3(b) the energy of the optimal cuts obtained by the heuristic MM 1 
(-A-) is lower than the one obtained by the maximal matching heuristic (-•-). This 
result is confirmed by Fig. 2 (lines MM and MM 1 ) where the heuristic MM removes 
more details of the mushroom at a given scale. This result is connected to the greater 
decimation ratio of the MM heuristic. The MM heuristic merges at each step regions 
with important scale of appearance without considering regions which may appear at 
further steps. The algorithms MM and MM 1 induce equivalent execution times on a 
sequential machine. The execution times of the method MM 1 (-A-) are overlay ed by 
the ones of the method MM (- • -) in Fig. 3(a) due to the vertical scale of this figure. 

The subjective quality of the partitions obtained by the heuristics MM 1 and SM 2 
(Fig. 2) seems roughly similar. We can notice that the heuristic MM 1 seems to produce 
slightly coarser partitions at each scale. However, considering Fig. 3(b), the optimal 
energy obtained by the heuristic S M 2 (-□-) are lower than the one obtained by MM 1 
(-A-). Note that the heuristic MM 1 produces lower execution times than S M 2 even on 
a sequential machine(Fig. 3(a)). 

As shown by Fig. 3(b) the optimal energies produced by the heuristic SM (- + -) 
are always below the one produced by the heuristic SM 2 (-□-). Note that, the curve 
(- + -) is close to the diagonal of the square [0, l] 2 . This last point indicates that on 
most of the images of the Berkeley database the hierarchies produced by the S M heuris- 
tic provide optimal cuts whose normalised energy is closed from the lower bound of the 
optimal cut's energies (equation 5). This result is confirmed by Fig. 2 where the heuris- 
tic SM preserves more details of the image at each scale. However, the heuristic SM 
is the one which requires the more important execution times on a sequential machine 
(Fig. 3(a)). 
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The heuristic S M 5 may be understood as a compromise between S M 2 and SM. As 
shown by Fig. 3(b) the optimal energies obtained by the heuristic S M 5 < _) are close to 
the one obtain by S M(- + -) and below the one obtained by SM 2 (-n-). Moreover, as 
shown by Fig. 3(a), the execution times required by S M 5 are between the one required 
by the heuristics S M 2 and S M. Finally, the partitions obtained by the S M 5 heuristic in 
Fig. 2 are closed from the one obtained by the heuristic SM. 

Fig. 4 shows results obtained using an other fit to data criterion based on the intu- 
itive notion of contrast. The basic idea of this criterion [11] states that a region should 
have a higher contrast with its neighbours (called external contrast) than within its even- 
tual subparts (called internal contrast). Let us denote by G e the mean gradient computed 
along the contour associated to an edge e. The internal and external contrasts of a region 
R are then respectively defined as Int(R) = max ee cc{R)G e and Ext(R) = min eE E\vei(e)G e . 
Where CC(R) denotes the set of edges which have been contracted to define R and 
e e E\v e i(e) denotes the set of edges incident to v. Our new energy combines the con- 
trast and the squared error criteria as follows: 



where /() denotes a sigmoid function. 

A contrasted region will thus have a low ratio between its internal and external con- 
trast. Conversely, a poorly contrasted region may have a fit to data term close to twice 
its squared error. As shown by Fig. 3(b) and (c) this energy favours highly contrasted 
regions. For example, the cloud merged with the sky in Fig. 3(b) remains in Fig. 3(c). 
Moreover, experiments not reported here, shown us that the same type of discussion 
about the advantages and drawbacks of the different heuristics may be conducted on 
this new energy with the same conclusions. 

5 Conclusion 

The Scale Set framework is based on two steps: the determination of a hierarchy accord- 
ing to an energy criterion and the determination of optimal cuts within this hierarchy. 
We have presented in this article parallel and sequential heuristics to build such hierar- 
chies. The normalised energy of the optimal cuts, associated with these hierarchy are 
bounded bellow by the diagonal of the unit square [0, l] 2 . Our experimental results sug- 
gest that our sequential heuristic S M provides hierarchies whose normalised energies 
are closed from this lower bound. This methods may however require important exe- 
cution times. We thus propose an alternative heuristic providing lower execution time 
at the price of generally slightly higher optimal cut's energies. Our parallel methods 
provide greater energies than the one produced by Guigues's heuristic. However, these 
methods require less execution times even on sequential machine. 

Hierarchies encoding a sequence of optimal cuts are usually composed of a lower 
number of levels and regions than the initial hierarchies built by our merge heuristics. 
In the future, we would like to use these hierarchies of optimal cuts in order to match 




(6) 
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two hierarchies encoding the content of two images sharing a significant part of a same 
scene. 
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